{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.网络结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注：原图应该是227 * 227，否则stride为4，卷积核大小为11，padding为0得不到55 * 55的特征图"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![在这里插入图片描述](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/dbcbcc632ea823bc331e7fae8d2790ff.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 卷积层C1\n",
    "\n",
    "    C1的基本结构为：卷积–>ReLU–>池化\n",
    "\n",
    "- 卷积：输入227×227×3，96个11×11×3的卷积核，padding = 0，步长stride = 4，因此其FeatureMap大小为(227-11+0×2)/4+1 = 55，即55×55×96;\n",
    "- 激活函数：ReLU；\n",
    "- 池化：池化核大小3×3，不扩充边缘padding = 0，步长stride = 2，因此其FeatureMap输出大小为(55-3+0×2)/2+1=27, 即C1输出为27×27×96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 卷积层C2\n",
    "\n",
    "    C2的基本结构为：卷积–>ReLU–>池化\n",
    "\n",
    "- 卷积：输入27×27×96，256个5×5×96的卷积核，padding = 2，步长stride = 1，因此其FeatureMap大小为(27-5+2*2)/1+1 = 27，即27×27×256;\n",
    "- 激活函数：ReLU；\n",
    "- 池化：池化核大小3×3，padding = 0，步长stride = 2，因此其FeatureMap输出大小为(27-3)/2+1=13, 即C1输出为13×13×256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 卷积层C3\n",
    "\n",
    "    C3的基本结构为：卷积–>ReLU\n",
    "\n",
    "- 卷积：输入13×13×256，384个3×3×256的卷积核，padding = 1，步长stride = 1，因此其FeatureMap大小为13×13×384;\n",
    "- 激活函数：ReLU；"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 卷积层C4\n",
    "\n",
    "    C4的基本结构为：卷积–>ReLU\n",
    "\n",
    "- 卷积：输入13×13×384，384个3×3×384的卷积核， 扩充边缘padding = 1，步长stride = 1，因此其FeatureMap大小为(13-3+1×2+1)/1 = 13，即13×13×384;\n",
    "- 激活函数：ReLU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.5 卷积层C5\n",
    "\n",
    "    C5的基本结构为：卷积–>ReLU–>池化\n",
    "\n",
    "- 卷积：输入13×13×384，256个3×3×384的卷积核，扩充边缘padding = 1，步长stride = 1，因此其FeatureMap大小为(13-3+1×2+1)/1 = 13，即13×13×256;\n",
    "- 激活函数：ReLU；\n",
    "- 池化：池化核大小3 × 3， 扩充边缘padding = 0，步长stride = 2，因此其FeatureMap输出大小为(13-3+0×2+2)/2=6, 即C5输出为6×6×256"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.6 全连接层FC6\n",
    "\n",
    "    FC6的基本结构为：全连接–>>ReLU–>Dropout\n",
    "\n",
    "- 全连接：输入个数为6×6×256，输出个数为4096\n",
    "- 激活函数：ReLU；\n",
    "- Dropout：全连接层中去掉了一些神经节点，达到防止过拟合，FC6输出为1×1×4096；\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.7 全连接层FC7\n",
    "\n",
    "    FC7的基本结构为：全连接–>>ReLU–>Dropout\n",
    "\n",
    "- 全连接：此层的全连接，输入1×1×4096;\n",
    "- 激活函数：ReLU；\n",
    "- Dropout：全连接层中去掉了一些神经节点，达到防止过拟合，FC7输出为1×1×4096；\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.8 全连接层FC8\n",
    "\n",
    "    FC8的基本结构为：全连接–>>softmax\n",
    "\n",
    "- 全连接：此层的全连接，输入1×1×4096;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.代码实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 1, loss:0.01799769111474355, accuracy:0.1296\n",
      "epoch: 2, loss:0.017746040868759156, accuracy:0.4056\n",
      "epoch: 3, loss:0.01458304424683253, accuracy:0.6766\n",
      "epoch: 4, loss:0.013494119010368982, accuracy:0.749\n",
      "epoch: 5, loss:0.013120386290550233, accuracy:0.7838\n",
      "epoch: 6, loss:0.01290577528278033, accuracy:0.8197\n",
      "epoch: 7, loss:0.012790821182727813, accuracy:0.8141\n",
      "epoch: 8, loss:0.012667479821046193, accuracy:0.838\n",
      "epoch: 9, loss:0.012604453392823537, accuracy:0.8211\n",
      "epoch: 10, loss:0.012553473156690597, accuracy:0.8463\n",
      "epoch: 11, loss:0.01250447180668513, accuracy:0.8538\n",
      "epoch: 12, loss:0.012467773564656575, accuracy:0.8561\n",
      "epoch: 13, loss:0.012435130725304286, accuracy:0.8621\n",
      "epoch: 14, loss:0.012389210832118988, accuracy:0.8592\n",
      "epoch: 15, loss:0.012355718980232874, accuracy:0.8734\n",
      "epoch: 16, loss:0.01232134511868159, accuracy:0.874\n",
      "epoch: 17, loss:0.012306503377358119, accuracy:0.8782\n",
      "epoch: 18, loss:0.012275078813234966, accuracy:0.8785\n",
      "epoch: 19, loss:0.012254926518599192, accuracy:0.8826\n",
      "epoch: 20, loss:0.012238973945379258, accuracy:0.8838\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA7AUlEQVR4nO3de3wU9b3/8fdukt2QeyDkSiCCICJXiaSAVAvRoMWCtRWxB5Cj9OgDFEt7fkAr0B5Pja1KaSuVllO0fVgE9VS0QlHIAT0oitwqIoc7BAi5cMs92WR3fn/QrAnktskmk919PR+PeWwy853Zz2Rc983Md75jMQzDEAAAgEmsZhcAAAACG2EEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGCqYLMLaA2Xy6W8vDxFRkbKYrGYXQ4AAGgFwzBUWlqq5ORkWa1Nn//wiTCSl5en1NRUs8sAAABtcPr0afXq1avJ5T4RRiIjIyVd2ZmoqCiTqwEAAK1RUlKi1NRU9/d4U3wijNRdmomKiiKMAADgY1rqYkEHVgAAYCrCCAAAMBVhBAAAmMon+oy0htPpVE1NjdllwENBQUEKDg7mlm0ACGB+EUbKysp05swZGYZhdilog7CwMCUlJclms5ldCgDABD4fRpxOp86cOaOwsDD17NmTf2H7EMMw5HA4VFRUpBMnTqh///7NDooDAPBPPh9GampqZBiGevbsqW7dupldDjzUrVs3hYSE6NSpU3I4HAoNDTW7JABAJ/Obf4ZyRsR3cTYEAAIb3wIAAMBUhBEAAGAqwohJbr/9dj355JNmlwEAgOkIIwAAwFQ+fzcNAAD+yjAM1dTUqKqqSk6nUy6Xq8Nex4wZo5iYGFP20//CiGFIFRXmvHdYmNSGu3ouXbqkefPm6W9/+5uqq6t122236Te/+Y369+8vSTp16pTmzp2r7du3y+FwKC0tTc8995zuvvtuXbp0SXPnztX777+vsrIy9erVSz/+8Y81a9Ysb+8dAKAN6gJFZWWlqqqqmnxtapnL5eqUOocOHUoY8ZqKCikiwpz3LiuTwsM9Xu2hhx7SkSNH9M477ygqKkoLFizQ3XffrS+//FIhISGaM2eOHA6HPvzwQ4WHh+vLL79UxD/3cfHixfryyy/197//XXFxcTp69KgqKyu9vWcA4BMMw1Btba0qKysbfKFXVlaqtrbWPVJ3/demfvZ0XnV1dYcFCovFoqCgIAUFBclqtbbrtW5bV8+PjIxsV43t4X9hxMfUhZCPPvpIY8aMkST95S9/UWpqqtavX6/vfve7ys3N1X333achQ4ZIkvr27etePzc3VyNGjFB6erokKS0trdP3AQC8zeVyXRMmrv69qZ+dTqfZ5TfKarUqNDRU3bp1U7du3dw/h4aGNvi5sdeQkBC/Hk/L/8JIWNiVMxRmvbeHDh48qODgYGVkZLjn9ejRQzfccIMOHjwoSXriiSf02GOP6f3331dmZqbuu+8+DR06VJL02GOP6b777tOePXt05513asqUKe5QAwDtZRiGiouLdfr0aeXn57vPLhiGIZfL5f65/nT1/JZ+r5vndDrdgcLhcLSrbovF0uDLvFu3bu6HctZNde3qXhv7ubFlTbW32WwBHSjaw//CiMXSpkslXdkjjzyirKwsbdiwQe+//76ys7P1wgsv6PHHH9ddd92lU6dOaePGjdq8ebMmTJigOXPm6Pnnnze7bAA+yOVyKT8/X6dPn9bp06eVm5ur0tJS0+qx2+3XhIqrv+Svnt+tWzfZbDa++H2I/4URH3PjjTeqtrZWn376qfuMxoULF3To0CENGjTI3S41NVWPPvqoHn30US1atEirVq3S448/Lknq2bOnZs6cqZkzZ2rcuHH693//d8IIgFaprq7WmTNnlJubq9OnT+vMmTOqqalp0MZqtSoxMVEpKSmy2+2yWq0NzjB46/egoKBrQgePiwgMhBGT9e/fX5MnT9bs2bP1+9//XpGRkVq4cKFSUlI0efJkSdKTTz6pu+66SwMGDNClS5e0detW3XjjjZKkJUuWaOTIkbrppptUXV2td999170MAOqrf8mlLnwUFha6O2HWsdvtSk1NVWpqqnr37q3k5GTZbDaTqkYgIIx0AS+//LLmzZunSZMmyeFw6Otf/7o2btyokJAQSZLT6dScOXN05swZRUVFaeLEifrVr34lSbLZbFq0aJFOnjypbt26ady4cVq7dq2ZuwPgKoZhyOFwqLS0VE6nUzabzT3V9WPoCK295BITE6PevXu7A0h8fDyXONCpLMbVkbgLKikpUXR0tIqLixUVFdVgWVVVlU6cOKHrrruOx8/7KI4hfJVhGKqsrFRpaanKysquea3/89WXPuqz2Wyy2+0NQkrdFBIS0uj8xqbg4GAVFRW5w0dzl1zqznqkpqaaeksn/Ftz39/1cWYEAK7icrmuCRONBYyysjKPxo+w2+0KDg6Ww+FoEBIcDke77x5p7j3rX3JJSUlxn3UFugrCCACfUjeoVU1Njfu1/nT1vObaNLbM4XCovLzco5rCwsIUERGhyMhIRURENPi5/rz6/S7qRuWsCyJXT9XV1Y3Ob2ydurY1NTWKjo52n/Ho3bu3evbsySUXdHmEEQBdSm1trYqLi3X58uUGr3U/l5SUXNPhsiNYLJZrwkRTISMoKKhN26+7vOIthmEQPOCTCCMAOlV1dXWjYaPutcyDQQutVqtCQkIaTMHBwU3Ou3pZY21tNpsiIiIUFhbmc1/svlYvUIcwAsCrqqurdfHixSbDRmuenRQSEqKYmBhFR0crOjra/XPdq91uV0hICGNQAH6CMAKg3QzD0OnTp7Vr1y59+eWXLT4bJDQ09JqwUT9wdOvWjX/lAwGEMAKgzaqrq/X5559r165dKiwsdM8PDw+/5mxG/Ve73W5i1QC6GsIIAI/l5+dr165d+vzzz923qAYHB2vw4MFKT09XSkqKyRUC8CWEEaCLMAxDp06d0p49e5SXl6fk5GT17dtX/fr16xKDUtXW1urAgQPatWuXzpw5454fFxenkSNHatiwYerWrZuJFQLwVYQRwGSlpaXat2+f9u7dq0uXLrnnX7hwQfv375ckxcfHq1+/furXr5969+7dqYNWXbhwQbt379a+ffvcnU+tVqsGDhyo9PR0paWl0b8DQLsQRuBWU1PDyIydxOl06siRI9q7d6+OHDniHjfDZrNp8ODB6t+/v86ePavjx48rLy9PhYWFKiws1I4dOxQcHKw+ffq4z5p0xHNEXC6XDh06pF27dun48ePu+dHR0br55pt18803KyIiwqvvCSBwcV+ciTZt2qRbb71VMTEx6tGjhyZNmqRjx465l585c0bTpk1T9+7dFR4ervT0dH366afu5X/72990yy23KDQ0VHFxcbr33nvdyywWi9avX9/g/WJiYvTKK69Ikk6ePCmLxaJ169bptttuU2hoqP7yl7/owoULmjZtmlJSUhQWFqYhQ4botddea7Adl8ulX/7yl7r++utlt9vVu3dv/fznP5ckjR8/XnPnzm3QvqioSDabTTk5Od74s/m0CxcuaPPmzfrVr36ldevW6fDhwzIMQ71799bkyZP1wx/+UPfcc48GDhyoCRMmaPbs2fr3f/933XfffRo+fLgiIyNVW1urY8eOafPmzVq5cqWWLVum9evXa//+/R6PHHq1kpISbdu2TcuXL9frr7/uDiL9+/fXAw88oCeeeEJf//rXCSIAvMrvzozUDbFshpCQEI/+hVpeXq758+dr6NChKisr05IlS3Tvvfdq3759qqio0G233aaUlBS98847SkxM1J49e9zPwdiwYYPuvfde/eQnP9Gf//xnORwObdy40eOaFy5cqBdeeEEjRoxQaGioqqqqNHLkSC1YsEBRUVHasGGDpk+frn79+mnUqFGSpEWLFmnVqlX61a9+pVtvvVXnzp3T//3f/0mSHnnkEc2dO1cvvPCC+46JV199VSkpKRo/frzH9fmDmpoaffnll9qzZ49yc3Pd88PDwzVs2DCNGDFCcXFxTa4fFhamwYMHa/DgwTIMQ+fPn9exY8d07NgxnTx5UmVlZfrHP/6hf/zjH5KkpKQk9e3bV9dff71SU1NbHB3UMAwdP35cu3bt0qFDh9xnacLCwjRixAiNHDlSsbGxXvhLAEDj/O6pvQ6HQ9nZ2abUuWjRonYN7Xz+/Hn17NlT+/fv18cff6wf/ehHOnnypLp3735N2zFjxqhv37569dVXG92WxWLRW2+9pSlTprjnxcTEaPny5XrooYd08uRJXXfddVq+fLnmzZvXbF2TJk3SwIED9fzzz6u0tFQ9e/bUiy++qEceeeSatlVVVUpOTtbKlSt1//33S5KGDRumb3/721q6dGmj2/fHp/YahqFz585pz549+uKLL1RdXS3pynG5/vrrNWLECA0YMKBNw4jXV1tbq9zcXHc4KSgoaLA8JCREaWlp7v4mPXr0cAfmiooK7du3T7t379bFixfd6/Tp00fp6ekaOHCggoP97t8rADoRT+31AUeOHNGSJUv06aef6vz58+6zHrm5udq3b59GjBjRaBCRpH379mn27NntriE9Pb3B706nU88884xef/11nT171v0QrrCwMEnSwYMHVV1drQkTJjS6vdDQUE2fPl2rV6/W/fff7/4yfuedd9pdqy+orKzU559/rr179zYIBrGxsRo+fLiGDx/e7AfSU8HBwerbt6/69u2rO+64Q2VlZTp+/Lg7nJSXl+vIkSM6cuSIpCt9Pvr27Sun06kDBw64Byez2+0aOnSo0tPTFR8f77X6AKA1/C6MhISEaNGiRaa9tyfuuece9enTR6tWrVJycrJcLpcGDx4sh8PR4i2SLS23WCzXPEyssctX4eHhDX5/7rnn9Otf/1rLly/XkCFDFB4erieffNL9ePPW3Lr5yCOPaPjw4Tpz5oxefvlljR8/Xn369GlxPW/Jy8vTtm3bVFpaqoiICIWHhys8PNz9ULP6P3tjpE/DMHTixAnt3btXBw8edH/BBwUFadCgQRoxYkSn3XESERGhoUOHaujQoTIMQwUFBe5gkpubq+LiYu3du9fdPikpSenp6Ro8eLBXH9gGAJ7wuzBS9yTMru7ChQs6dOiQVq1apXHjxkmStm/f7l4+dOhQ/dd//ZcuXrzY6NmRoUOHKicnR7NmzWp0+z179tS5c+fcvx85ckQVFRUt1vXRRx9p8uTJ+pd/+RdJVzqrHj58WIMGDZJ0pSNjt27dlJOT0+hlGkkaMmSI0tPTtWrVKq1Zs0Yvvvhii+/rDeXl5crJyWnwZdsSi8VyTVBpbXApKSnR3r17tW/fPl2+fNk9PyEhQTfffLOGDBli6rgbFotFiYmJSkxM1NixY1VTU6NTp07p2LFjcjqdGjZsmJKTk7ktF4Dp/C6M+IrY2Fj16NFDf/jDH5SUlKTc3FwtXLjQvXzatGl65plnNGXKFGVnZyspKUl79+5VcnKyRo8eraVLl2rChAnq16+fHnjgAdXW1mrjxo1asGCBpCt3tbz44osaPXq0nE6nFixY0KozN/3799ebb76pjz/+WLGxsVq2bJkKCgrcYSQ0NFQLFizQ//t//082m01jx45VUVGRDhw4oIcffti9nbqOrOHh4Q3u8ukITqdTO3fu1AcffODumzFkyBANHjxY5eXlKi8vV1lZ2TWvlZWVMgxDZWVlrXpSrNVqVVhYmCIiIhQcHKyzZ8+6zz7Z7XYNGTJEI0aMUFJSUpf8gg8JCdH111+v66+/3uxSAKABwohJrFar1q5dqyeeeEKDBw/WDTfcoN/85je6/fbbJV0Zb+L999/XD3/4Q919992qra3VoEGDtGLFCknS7bffrjfeeENPP/20nn32WUVFRenrX/+6e/svvPCCZs2apXHjxik5OVm//vWvtXv37hbreuqpp3T8+HFlZWUpLCxM3//+9zVlyhQVFxe72yxevFjBwcFasmSJ8vLylJSUpEcffbTBdqZNm6Ynn3xS06ZN69BOqceOHdOmTZt0/vx5SVcuO0ycOFG9e/ducV2n09lkUKl7rfu5srJSLpfrmuDSp08fjRgxQoMGDWKMFgBoI7+7mwZdw8mTJ9WvXz999tlnuvnmm5tt25ZjePHiRb3//vs6dOiQpCu3oY4fP14jRozokMfKXx1cKisrlZKSoh49enj9vQDAX7T2bpo2/V97xYoVSktLU2hoqDIyMrRz585m2y9fvlw33HCDunXrptTUVP3gBz9QVVVVW94aXVxNTY3y8/P11FNP6Wtf+1qLQcRTDodDOTk5+t3vfqdDhw7JYrEoIyNDjz/+uEaOHNkhQUS60hk1KipKycnJ6t+/v4YOHUoQAQAv8fgyzbp16zR//nytXLlSGRkZWr58ubKysnTo0KFGbwlcs2aNFi5cqNWrV2vMmDE6fPiwHnroIVksFi1btswrO4Gu46OPPtI3vvENDRgwQG+++abXtmsYhvbv368tW7aotLRUktS3b19NnDhRPXv29Nr7AAA6n8dhZNmyZZo9e7b7Lo6VK1dqw4YNWr16dYMOmHU+/vhjjR07Vg8++KAkKS0tTdOmTWswrDn8x+23337NLcXtlZeXp7///e/uJ8XGxsbqzjvv1A033NAlO4oCADzjURhxOBzavXt3g3E8rFarMjMztWPHjkbXGTNmjF599VXt3LlTo0aN0vHjx7Vx40ZNnz69yfeprq523xUhXbnmhMBz9a26ISEhGjdunEaPHs3IoADgRzz6P/r58+fldDqVkJDQYH5CQoL72SRXe/DBB3X+/HndeuutMgxDtbW1evTRR/XjH/+4yffJzs7Wz372M09Kgx9p6lbdzMxMr45eCgDoGjr8qb3btm3TM888o9/97nfas2eP/vrXv2rDhg16+umnm1xn0aJFKi4udk+nT59u8X184KYgNKH+sTt69KhWrlyp999/X9XV1UpKStKsWbP07W9/myACAH7KozMjcXFxCgoKuuZhXAUFBUpMTGx0ncWLF2v69Onu0TqHDBmi8vJyff/739dPfvKTRu9+sNvt7ie+tqTuQWOtGUIdXVNFRYWcTqfWr1/f4FbdCRMmaPjw4R12hwwAoGvwKIzYbDaNHDlSOTk57qfBulwu5eTkaO7cuY2uU1FRcc2XSV2A8MbZjODgYIWFhamoqEghISF8cfmQutFPz549q4MHD+rw4cOyWq265ZZbdPvttzNuDAAECI97Ac6fP18zZ85Uenq6Ro0apeXLl6u8vNx9d82MGTOUkpKi7OxsSVceBrds2TKNGDFCGRkZOnr0qBYvXqx77rmn3Y9Pl648fyMpKUknTpzQqVOn2r09dB6Hw6Hy8nKdOHFCR48eVb9+/ZSVlcWtugAQYDwOI1OnTlVRUZGWLFmi/Px8DR8+XJs2bXJ3as3NzW1wduKpp56SxWLRU089pbNnz6pnz56655579POf/9xrO2Gz2dS/f3/3k2XRteXn52v79u3Kz89XVVWVoqKiNHXqVG7VBYAA5fPDwcN3FBcXa8uWLfriiy8kcasuAPi71n5/8w2ADlddXa3t27frk08+UW1trSRp+PDhGj9+vCIjI02uDgBgNsIIOozL5dK+ffu0detW95Nu+/Tpo6ysLCUlJZlcHQCgqyCMoEOcOHFC7733nvs28NjYWN1xxx0aOHAg/UIAAA0QRuBVFy5c0ObNm93jhdjtdt12220aNWqUV+6eAgD4H8IIvKKyslIffPCBPvvsM7lcLlksFqWnp+v2229XWFiY2eUBALowwgjaxel0ateuXfrggw9UWVkpSerfv7/uuOMOxgsBALQKYQRtYhiGDh8+rM2bN+vChQuSpPj4eN15553q16+fydUBAHwJYQQeKygo0HvvvacTJ05IuvIcmW984xu6+eabGY4fAOAxwgharaysTFu3btXevXtlGIaCgoL0ta99TbfeeivPkQEAtBlhBC2qra3Vjh07tH37dveQ+4MGDVJmZqZiY2NNrg4A4OsII2iSYRg6cOCAtmzZouLiYklScnKysrKy1Lt3b5OrAwD4C8IIGnXhwgW9/fbbOn36tCQpKipKEyZM0JAhQxi0DADgVYQRXKO8vFyvvvqqLl++rJCQEI0dO1ZjxoxRSEiI2aUBAPwQYQQN1NbWau3atbp8+bJiY2P10EMP8aRkAECH4j5MuBmGobfffltnzpxRaGioHnzwQYIIAKDDEUbgtm3bNn3xxReyWq26//77FRcXZ3ZJAIAAQBiBJOnzzz/Xhx9+KEmaNGmSrrvuOpMrAgAECsIIlJubq3feeUeSNHbsWI0YMcLkigAAgYQwEuAuXryotWvXyul06sYbb9SECRPMLgkAEGAIIwGssrJSr732miorK5WcnKx7772XMUQAAJ2OMBKgnE6n3njjDZ0/f15RUVF64IEHGEcEAGAKwkgAMgxDGzZs0IkTJ2Sz2TRt2jRFRkaaXRYAIEARRgLQjh07tHfvXlksFt13331KTEw0uyQAQAAjjASYgwcPavPmzZKkrKwsDRgwwOSKAACBjjASQPLy8vTXv/5VknTLLbdo1KhRJlcEAABhJGCUlJTotddeU21tra6//npNnDiRO2cAAF0CYSQAOBwOrVmzRmVlZYqPj9d3vvMdWa0cegBA18A3kp9zuVz67//+bxUUFCg8PFzTpk2T3W43uywAANwII35u8+bNOnz4sIKDg/XAAw8oJibG7JIAAGiAMOLHPvvsM33yySeSpClTpqhXr14mVwQAwLUII37q6NGj+vvf/y5JGj9+vG666SaTKwIAoHGEET9UWFioN998U4ZhaNiwYbr11lvNLgkAgCYRRvxMWVmZ1qxZo+rqavXp00eTJk3iFl4AQJdGGPEjNTU1WrdunYqLi9W9e3fdf//9Cg4ONrssAACaRRjxE4Zh6O2339aZM2cUGhqqBx98UGFhYWaXBQBAiwgjfmLbtm06cOCArFarpk6dqh49ephdEgAArUIY8QP/+Mc/9OGHH0qSJk2apLS0NHMLAgDAA4QRH3fq1Cn97W9/kySNHTtWI0aMMLkiAAA8QxjxYRcvXtS6devkdDp14403asKECWaXBACAxwgjPqq6ulpr1qxRZWWlkpOTde+993ILLwDAJxFGfNTBgwd14cIFRUZGatq0aQoJCTG7JAAA2oQw4qNOnz4tSRoyZIgiIiJMrgYAgLYjjPioujCSmppqciUAALQPYcQHVVVVqaioSJJ4Ei8AwOcRRnzQmTNnJEndu3fnEg0AwOcRRnwQl2gAAP6EMOKD6sIIl2gAAP6AMOJjXC6Xzp49K4kzIwAA/0AY8TEFBQVyOByy2+3q2bOn2eUAANBuhBEfU/8SjdXK4QMA+D6+zXxM3Z009BcBAPgLwoiPqTsz0rt3b5MrAQDAOwgjPqS0tFSXL1+WxWJRSkqK2eUAAOAVhBEfUndWJD4+Xna73eRqAADwDsKID2GwMwCAPyKM+BDCCADAHxFGfERNTY3OnTsniTACAPAvhBEfce7cOblcLkVERCgmJsbscgAA8BrCiI+of4nGYrGYXA0AAN5DGPERPBwPAOCvCCM+wDAMOq8CAPwWYcQHXLp0SRUVFQoKClJSUpLZ5QAA4FWEER+Qm5srSUpOTlZwcLDJ1QAA4F2EER9AfxEAgD8jjPiAuif18nA8AIA/Iox0cVVVVSosLJTEmREAgH9qUxhZsWKF0tLSFBoaqoyMDO3cubPZ9pcvX9acOXOUlJQku92uAQMGaOPGjW0qONDUnRWJjY1VRESEydUAAOB9HveGXLdunebPn6+VK1cqIyNDy5cvV1ZWlg4dOqT4+Phr2jscDt1xxx2Kj4/Xm2++qZSUFJ06dYpRRFuJW3oBAP7O4zCybNkyzZ49W7NmzZIkrVy5Uhs2bNDq1au1cOHCa9qvXr1aFy9e1Mcff6yQkBBJUlpaWvuqDiB1Z0YIIwAAf+XRZRqHw6Hdu3crMzPzqw1YrcrMzNSOHTsaXeedd97R6NGjNWfOHCUkJGjw4MF65pln5HQ6m3yf6upqlZSUNJgCkcvlIowAAPyeR2Hk/PnzcjqdSkhIaDA/ISFB+fn5ja5z/Phxvfnmm3I6ndq4caMWL16sF154Qf/5n//Z5PtkZ2crOjraPQXqF3FhYaEcDofsdrt69uxpdjkAAHSIDr+bxuVyKT4+Xn/4wx80cuRITZ06VT/5yU+0cuXKJtdZtGiRiouL3VNdv4lAU398EauVG58AAP7Joz4jcXFxCgoKUkFBQYP5BQUFSkxMbHSdpKQkhYSEKCgoyD3vxhtvVH5+vhwOh2w22zXr2O122e12T0rzSwx2BgAIBB79c9tms2nkyJHKyclxz3O5XMrJydHo0aMbXWfs2LE6evSoXC6Xe97hw4eVlJTUaBDBV7iTBgAQCDw+9z9//nytWrVKf/rTn3Tw4EE99thjKi8vd99dM2PGDC1atMjd/rHHHtPFixc1b948HT58WBs2bNAzzzyjOXPmeG8v/FBpaakuX74siTMjAAD/5vGtvVOnTlVRUZGWLFmi/Px8DR8+XJs2bXJ3as3NzW3QvyE1NVXvvfeefvCDH2jo0KFKSUnRvHnztGDBAu/thR+qOyuSkJDAJSsAgF+zGIZhmF1ES0pKShQdHa3i4mJFRUWZXU6neO+99/TJJ59o5MiRmjRpktnlAADgsdZ+f3OLRhfFw/EAAIGCMNIF1dbWKi8vTxKdVwEA/o8w0gXl5eXJ5XIpPDycZ/gAAPweYaQLqn9Lr8ViMbkaAAA6FmGkC+J5NACAQEIY6WIMw1Bubq4kwggAIDAQRrqYS5cuqaKiQkFBQUpKSjK7HAAAOhxhpIup6y+SnJys4GCPx6QDAMDnEEa6GB6OBwAINISRLoaH4wEAAg1hpAupqqpSYWGhJMIIACBwEEa6kLNnz0qSYmNjFRERYXI1AAB0DsJIF8ItvQCAQEQY6UIY7AwAEIgII12Ey+UijAAAAhJhpIsoLCyUw+GQzWZTz549zS4HAIBOQxjpIuqPL2K1clgAAIGDb70ugks0AIBARRjpIhjsDAAQqAgjXUBZWZkuXbokiWHgAQCBhzDSBdSdFUlISJDdbje5GgAAOhdhpAvg4XgAgEBGGOkC6C8CAAhkhBGT1dbW6ty5c5IIIwCAwEQYMdm5c+fkdDoVHh6u2NhYs8sBAKDTEUZMVv8SjcViMbkaAAA6H2HEZPQXAQAEOsKIiQzDIIwAAAIeYcREly5dUnl5uYKCgpSUlGR2OQAAmIIwYqK6syJJSUkKDg42uRoAAMxBGDERl2gAACCMmIon9QIAQBgxTVVVlQoKCiQRRgAAgY0wYpKzZ89KkmJjYxUREWFyNQAAmIcwYhL6iwAAcAVhxCQ8qRcAgCsIIyZwuVx0XgUA4J8IIyYoKiqSw+GQzWZTfHy82eUAAGAqwogJ6l+isVo5BACAwMY3oQnovAoAwFcIIyYgjAAA8BXCSCcrKyvTpUuXJEkpKSkmVwMAgPkII52s7qxIfHy8QkNDTa4GAADzEUY6GZdoAABoiDDSyRhfBACAhggjnai2tlZ5eXmSCCMAANQhjHSic+fOyel0Kjw8XLGxsWaXAwBAl0AY6UT1+4tYLBaTqwEAoGsgjHQiHo4HAMC1CCOdxDAM7qQBAKARhJFOcvnyZZWXl8tqtSo5OdnscgAA6DIII52k7qxIcnKygoODTa4GAICugzDSSbhEAwBA4wgjnYQwAgBA4wgjnaC6uloFBQWSuJMGAICrEUY6Qd0Q8DExMYqMjDS5GgAAuhbCSCfgEg0AAE0jjHQCHo4HAEDTCCMdzOVyEUYAAGgGYaSDFRUVqbq6WjabTfHx8WaXAwBAl0MY6WD1n0djtfLnBgDganw7djAejgcAQPMIIx2MO2kAAGgeYaQDlZWV6dKlS5I4MwIAQFMIIx2o7i6a+Ph4hYaGmlwNAABdE2GkA3GJBgCAlrUpjKxYsUJpaWkKDQ1VRkaGdu7c2ar11q5dK4vFoilTprTlbX0OYQQAgJZ5HEbWrVun+fPna+nSpdqzZ4+GDRumrKwsFRYWNrveyZMn9aMf/Ujjxo1rc7G+pLa2Vnl5eZIIIwAANMfjMLJs2TLNnj1bs2bN0qBBg7Ry5UqFhYVp9erVTa7jdDr1ve99Tz/72c/Ut2/fdhXsK/Lz8+V0OhUWFqbY2FizywEAoMvyKIw4HA7t3r1bmZmZX23AalVmZqZ27NjR5Hr/8R//ofj4eD388MOtep/q6mqVlJQ0mHxN/fFFLBaLydUAANB1eRRGzp8/L6fTqYSEhAbzExISlJ+f3+g627dv1x//+EetWrWq1e+TnZ2t6Oho9+SLlznq7qThll4AAJrXoXfTlJaWavr06Vq1apXi4uJavd6iRYtUXFzsnurOMvgSOq8CANA6wZ40jouLU1BQkAoKChrMLygoUGJi4jXtjx07ppMnT+qee+5xz3O5XFfeODhYhw4dUr9+/a5Zz263y263e1Jal1JcXKzS0lJZLBYlJyebXQ4AAF2aR2dGbDabRo4cqZycHPc8l8ulnJwcjR49+pr2AwcO1P79+7Vv3z739K1vfUvf+MY3tG/fPr89a1B3iSYxMVE2m83kagAA6No8OjMiSfPnz9fMmTOVnp6uUaNGafny5SovL9esWbMkSTNmzFBKSoqys7MVGhqqwYMHN1g/JiZGkq6Z7094OB4AAK3ncRiZOnWqioqKtGTJEuXn52v48OHatGmTu1Nrbm6urNbAHtiVzqsAALSexTAMw+wiWlJSUqLo6GgVFxcrKirK7HKaVVtbq+zsbLlcLj3xxBOMMQIACFit/f4O7FMYHeDcuXNyuVwKDw93X5ICAABNI4x4GYOdAQDgGcKIl9FfBAAAzxBGvMgwDHcY8dfblgEA8DbCiBfVDXZmtVoZ7AwAgFYijHhR/cHOQkJCTK4GAADfQBjxIgY7AwDAc4QRL6LzKgAAniOMeElNTY3y8/Ml0XkVAABPEEa8pG6ws4iICEVHR5tdDgAAPoMw4iUMdgYAQNsQRryE/iIAALQNYcQLGOwMAIC2I4x4QXFxscrKymS1WpWUlGR2OQAA+BTCiBfU9RdhsDMAADxHGPGCujDCJRoAADxHGPECOq8CANB2hJF2qqmpUUFBgSTOjAAA0BaEkXbKy8uTy+VSZGSkoqKizC4HAACfQxhpJwY7AwCgfQgj7UR/EQAA2ocw0g4MdgYAQPsRRtrh8uXLKi8vZ7AzAADagTDSDnX9RZKSkhQcHGxyNQAA+CbCSDvU77wKAADahjDSDvQXAQCg/QgjbeRwOBjsDAAALyCMtFFeXp4Mw1BUVBSDnQEA0A6EkTaivwgAAN5BGGkjBjsDAMA7CCNtwGBnAAB4D2GkDS5duqSKigoFBQUpMTHR7HIAAPBphJE2YLAzAAC8hzDSBnReBQDAewgjbUB/EQAAvIcw4qHq6moVFhZK4swIAADeQBjxUN1gZ9HR0Qx2BgCAFxBGPER/EQAAvIsw4iEGOwMAwLsIIx5gsDMAALyPMOKBixcvqrKyUsHBwQx2BgCAlxBGPFB/sLOgoCCTqwEAwD8QRjxAfxEAALyPMOKBujMj9BcBAMB7CCOtxGBnAAB0DMJIK509e1aSFB0drcjISJOrAQDAfxBGWolLNAAAdAzCSCvReRUAgI5BGGkFBjsDAKDjEEZa4cKFC6qqqlJwcLASEhLMLgcAAL9CGGmFuv4iycnJDHYGAICXEUZagf4iAAB0HMJIK9BfBACAjkMYaUFVVRWDnQEA0IEIIy2oG+wsJiZGERERJlcDAID/IYy0gMHOAADoWISRFtB5FQCAjkUYaQaDnQEA0PEII804f/68qqurFRISwmBnAAB0EMJIM+oPdma18qcCAKAj8A3bDPqLAADQ8QgjzaC/CAAAHY8w0oSqqioVFRVJ4swIAAAdiTDShLqzIrGxsQoPDze5GgAA/BdhpAkMdgYAQOcgjDSBzqsAAHSONoWRFStWKC0tTaGhocrIyNDOnTubbLtq1SqNGzdOsbGxio2NVWZmZrPtuwLDMNzPpCGMAADQsTwOI+vWrdP8+fO1dOlS7dmzR8OGDVNWVpb7ybZX27Ztm6ZNm6atW7dqx44dSk1N1Z133un+su+KioqKGOwMAIBOYjEMw/BkhYyMDN1yyy168cUXJUkul0upqal6/PHHtXDhwhbXdzqdio2N1YsvvqgZM2a06j1LSkoUHR2t4uJiRUVFeVJum+zevVvvvvuu0tLSNHPmzA5/PwAA/FFrv789OjPicDi0e/duZWZmfrUBq1WZmZnasWNHq7ZRUVGhmpoade/evck21dXVKikpaTB1JvqLAADQeTwKI+fPn5fT6bzm0kVCQoLy8/NbtY0FCxYoOTm5QaC5WnZ2tqKjo91TZ9/RwmBnAAB0nk69m+bZZ5/V2rVr9dZbbyk0NLTJdosWLVJxcbF7qrvNtjNUVlbq/PnzkjgzAgBAZwj2pHFcXJyCgoJUUFDQYH5BQYESExObXff555/Xs88+qy1btmjo0KHNtrXb7bLb7Z6U5jV1Z0W6d++usLAwU2oAACCQeHRmxGazaeTIkcrJyXHPc7lcysnJ0ejRo5tc75e//KWefvppbdq0Senp6W2vthMw2BkAAJ3LozMjkjR//nzNnDlT6enpGjVqlJYvX67y8nLNmjVLkjRjxgylpKQoOztbkvSLX/xCS5Ys0Zo1a5SWlubuWxIREaGIiAgv7op30HkVAIDO5XEYmTp1qoqKirRkyRLl5+dr+PDh2rRpk7tTa25urqzWr064vPTSS3I4HPrOd77TYDtLly7VT3/60/ZV72Uul4vBzgAA6GQehxFJmjt3rubOndvosm3btjX4/eTJk215C1MUFRXJ4XDIZrMpPj7e7HIAAAgIPJumnrr+IikpKQ3O7gAAgI7DN2499BcBAKDzEUbqYbAzAAA6H2HknyoqKnThwgVJnBkBAKAzEUb+qe6sSI8ePdStWzeTqwEAIHAQRv6Jwc4AADAHYeSf6LwKAIA5CCNisDMAAMxEGJFUWFiompoa2Ww29ezZ0+xyAAAIKIQRfdVfpFevXgx2BgBAJ+ObV/QXAQDATIQREUYAADBTwIeR8vJyXbx4URJhBAAAMwR8GKk7KxIXF8dgZwAAmIAwwiUaAABMFfBhhJFXAQAwV0CHEZfLpby8PEmcGQEAwCwBHUYKCgpUU1Mju93OYGcAAJgkoMNI/cHOLBaLydUAABCYAjqM0HkVAADzEUZEGAEAwEzBZhdgpunTp+vMmTPcSQMAgIkCOozExsYqNjbW7DIAAAhoAX2ZBgAAmI8wAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGCqYLMLMNUjj0iffy5ZLFcmq/Wrn7012e1SZKQUFXXltbHp6mXh4VdqAQAgAAR2GDlwQPrsM7OruJbFciWQtBRiIiOl0FApOFgKCWn6tbllLbWpC2hWa8Of6wcuAADaIbDDyHPPSZcuSYbh/cnluvJaXS2Vln41lZQ0/P3qeXXrlZVdmc6dM/uv1LLmAktrwkxTk9S+5fWDUv32rXltyzod9Xr1z80ta2271vzennlNhVRfn99Ryzvjb9/Ryzvrb98eZv1tfKX9D34gpaV5to6XBHYYufVWsytoyDCkysrWhZa6yeGQamqk2lrvvdbWela3y3Xl1en0/t8EANA5pk3zrTCyYsUKPffcc8rPz9ewYcP029/+VqNGjWqy/RtvvKHFixfr5MmT6t+/v37xi1/o7rvvbnPRfstikcLCrkwJCebVYRhXgkVNTcOzPC5Xw5+bevVkWVNTXR1tXV6/jS+/1j8mnfl7e+aZsa3OWN6R8zv6PTt6eWfV3x5m/W18pb0kpaR4vo6XeBxG1q1bp/nz52vlypXKyMjQ8uXLlZWVpUOHDik+Pv6a9h9//LGmTZum7OxsTZo0SWvWrNGUKVO0Z88eDR482Cs7AS+zWK70FwkO7BNnAIDOYTEMz+JTRkaGbrnlFr344ouSJJfLpdTUVD3++ONauHDhNe2nTp2q8vJyvfvuu+55X/va1zR8+HCtXLmyVe9ZUlKi6OhoFRcXKyoqypNyAQCASVr7/e3R/aMOh0O7d+9WZmbmVxuwWpWZmakdO3Y0us6OHTsatJekrKysJttLUnV1tUpKShpMAADAP3kURs6fPy+n06mEq/ozJCQkKD8/v9F18vPzPWovSdnZ2YqOjnZPqampnpQJAAB8SJccWWvRokUqLi52T6dPnza7JAAA0EE86qEYFxenoKAgFRQUNJhfUFCgxMTERtdJTEz0qL0k2e122e12T0oDAAA+yqMzIzabTSNHjlROTo57nsvlUk5OjkaPHt3oOqNHj27QXpI2b97cZHsAABBYPL53c/78+Zo5c6bS09M1atQoLV++XOXl5Zo1a5YkacaMGUpJSVF2drYkad68ebrtttv0wgsv6Jvf/KbWrl2rXbt26Q9/+IN39wQAAPgkj8PI1KlTVVRUpCVLlig/P1/Dhw/Xpk2b3J1Uc3NzZa33kLcxY8ZozZo1euqpp/TjH/9Y/fv31/r16xljBAAASGrDOCNmYJwRAAB8T4eMMwIAAOBthBEAAGAqwggAADAVYQQAAJjKJx7LWtfHlmfUAADgO+q+t1u6V8Ynwkhpaakk8YwaAAB8UGlpqaKjo5tc7hO39rpcLuXl5SkyMlIWi8Vr2y0pKVFqaqpOnz4dELcMB9L+sq/+K5D2l331X4Gyv4ZhqLS0VMnJyQ3GILuaT5wZsVqt6tWrV4dtPyoqyq//Y7haIO0v++q/Aml/2Vf/FQj729wZkTp0YAUAAKYijAAAAFMFdBix2+1aunSp7Ha72aV0ikDaX/bVfwXS/rKv/ivQ9rclPtGBFQAA+K+APjMCAADMRxgBAACmIowAAABTEUYAAICp/D6MrFixQmlpaQoNDVVGRoZ27tzZbPs33nhDAwcOVGhoqIYMGaKNGzd2UqXtk52drVtuuUWRkZGKj4/XlClTdOjQoWbXeeWVV2SxWBpMoaGhnVRx2/30pz+9pu6BAwc2u46vHte0tLRr9tVisWjOnDmNtve1Y/rhhx/qnnvuUXJysiwWi9avX99guWEYWrJkiZKSktStWzdlZmbqyJEjLW7X0899Z2huX2tqarRgwQINGTJE4eHhSk5O1owZM5SXl9fsNtvyWegMLR3Xhx566Jq6J06c2OJ2u+JxlVre38Y+wxaLRc8991yT2+yqx7aj+HUYWbdunebPn6+lS5dqz549GjZsmLKyslRYWNho+48//ljTpk3Tww8/rL1792rKlCmaMmWKvvjii06u3HMffPCB5syZo08++USbN29WTU2N7rzzTpWXlze7XlRUlM6dO+eeTp061UkVt89NN93UoO7t27c32daXj+tnn33WYD83b94sSfrud7/b5Dq+dEzLy8s1bNgwrVixotHlv/zlL/Wb3/xGK1eu1Keffqrw8HBlZWWpqqqqyW16+rnvLM3ta0VFhfbs2aPFixdrz549+utf/6pDhw7pW9/6Vovb9eSz0FlaOq6SNHHixAZ1v/baa81us6seV6nl/a2/n+fOndPq1atlsVh03333NbvdrnhsO4zhx0aNGmXMmTPH/bvT6TSSk5ON7OzsRtvff//9xje/+c0G8zIyMox/+7d/69A6O0JhYaEhyfjggw+abPPyyy8b0dHRnVeUlyxdutQYNmxYq9v703GdN2+e0a9fP8PlcjW63FePqWEYhiTjrbfecv/ucrmMxMRE47nnnnPPu3z5smG3243XXnutye14+rk3w9X72pidO3cakoxTp0412cbTz4IZGtvXmTNnGpMnT/ZoO75wXA2jdcd28uTJxvjx45tt4wvH1pv89syIw+HQ7t27lZmZ6Z5ntVqVmZmpHTt2NLrOjh07GrSXpKysrCbbd2XFxcWSpO7duzfbrqysTH369FFqaqomT56sAwcOdEZ57XbkyBElJyerb9+++t73vqfc3Nwm2/rLcXU4HHr11Vf1r//6r80+MNJXj+nVTpw4ofz8/AbHLjo6WhkZGU0eu7Z87ruq4uJiWSwWxcTENNvOk89CV7Jt2zbFx8frhhtu0GOPPaYLFy402dafjmtBQYE2bNighx9+uMW2vnps28Jvw8j58+fldDqVkJDQYH5CQoLy8/MbXSc/P9+j9l2Vy+XSk08+qbFjx2rw4MFNtrvhhhu0evVqvf3223r11Vflcrk0ZswYnTlzphOr9VxGRoZeeeUVbdq0SS+99JJOnDihcePGqbS0tNH2/nJc169fr8uXL+uhhx5qso2vHtPG1B0fT45dWz73XVFVVZUWLFigadOmNfsQNU8/C13FxIkT9ec//1k5OTn6xS9+oQ8++EB33XWXnE5no+395bhK0p/+9CdFRkbq29/+drPtfPXYtpVPPLUXnpkzZ46++OKLFq8vjh49WqNHj3b/PmbMGN144436/e9/r6effrqjy2yzu+66y/3z0KFDlZGRoT59+uj1119v1b82fNUf//hH3XXXXUpOTm6yja8eU3ylpqZG999/vwzD0EsvvdRsW1/9LDzwwAPun4cMGaKhQ4eqX79+2rZtmyZMmGBiZR1v9erV+t73vtdix3JfPbZt5bdnRuLi4hQUFKSCgoIG8wsKCpSYmNjoOomJiR6174rmzp2rd999V1u3blWvXr08WjckJEQjRozQ0aNHO6i6jhETE6MBAwY0Wbc/HNdTp05py5YteuSRRzxaz1ePqST38fHk2LXlc9+V1AWRU6dOafPmzR4/Wr6lz0JX1bdvX8XFxTVZt68f1zr/+7//q0OHDnn8OZZ899i2lt+GEZvNppEjRyonJ8c9z+VyKScnp8G/HOsbPXp0g/aStHnz5ibbdyWGYWju3Ll666239D//8z+67rrrPN6G0+nU/v37lZSU1AEVdpyysjIdO3asybp9+bjWefnllxUfH69vfvObHq3nq8dUkq677jolJiY2OHYlJSX69NNPmzx2bfncdxV1QeTIkSPasmWLevTo4fE2WvosdFVnzpzRhQsXmqzbl49rfX/84x81cuRIDRs2zON1ffXYtprZPWg70tq1aw273W688sorxpdffml8//vfN2JiYoz8/HzDMAxj+vTpxsKFC93tP/roIyM4ONh4/vnnjYMHDxpLly41QkJCjP3795u1C6322GOPGdHR0ca2bduMc+fOuaeKigp3m6v392c/+5nx3nvvGceOHTN2795tPPDAA0ZoaKhx4MABM3ah1X74wx8a27ZtM06cOGF89NFHRmZmphEXF2cUFhYahuFfx9Uwrtw10Lt3b2PBggXXLPP1Y1paWmrs3bvX2Lt3ryHJWLZsmbF37173HSTPPvusERMTY7z99tvG559/bkyePNm47rrrjMrKSvc2xo8fb/z2t791/97S594sze2rw+EwvvWtbxm9evUy9u3b1+AzXF1d7d7G1fva0mfBLM3ta2lpqfGjH/3I2LFjh3HixAljy5Ytxs0332z079/fqKqqcm/DV46rYbT837FhGEZxcbERFhZmvPTSS41uw1eObUfx6zBiGIbx29/+1ujdu7dhs9mMUaNGGZ988ol72W233WbMnDmzQfvXX3/dGDBggGGz2YybbrrJ2LBhQydX3DaSGp1efvlld5ur9/fJJ590/20SEhKMu+++29izZ0/nF++hqVOnGklJSYbNZjNSUlKMqVOnGkePHnUv96fjahiG8d577xmSjEOHDl2zzNeP6datWxv977Zun1wul7F48WIjISHBsNvtxoQJE675O/Tp08dYunRpg3nNfe7N0ty+njhxosnP8NatW93buHpfW/osmKW5fa2oqDDuvPNOo2fPnkZISIjRp08fY/bs2deECl85robR8n/HhmEYv//9741u3boZly9fbnQbvnJsO4rFMAyjQ0+9AAAANMNv+4wAAADfQBgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKn+P/JMQ9puqiG2AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "import torch.nn.functional as F\n",
    "from torchvision import transforms\n",
    "import torchsummary\n",
    "import Module.Utils as myutils\n",
    "\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "\n",
    "class AlexNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(AlexNet, self).__init__()\n",
    "        self.features = nn.Sequential(\n",
    "            nn.Conv2d(in_channels=1, out_channels=96, kernel_size=11, stride=4),    # FMNIST数据集中全是单通道灰度图，所以输入通道数为1\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=3, stride=2, padding=0),\n",
    "            nn.Conv2d(in_channels=96, out_channels=256, kernel_size=5, stride=1, padding=2),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=3, stride=2, padding=0),\n",
    "            nn.Conv2d(in_channels=256, out_channels=384, kernel_size=3, stride=1, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.Conv2d(in_channels=384, out_channels=384, kernel_size=3, stride=1, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.Conv2d(in_channels=384, out_channels=256, kernel_size=3, stride=1, padding=1),\n",
    "            nn.ReLU(),\n",
    "            nn.MaxPool2d(kernel_size=3, stride=2, padding=0),\n",
    "            nn.Flatten()\n",
    "        )\n",
    "\n",
    "        self.classifier = nn.Sequential(\n",
    "            nn.Linear(in_features=6*6*256, out_features=4096),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(),\n",
    "            nn.Linear(in_features=4096, out_features=4096),\n",
    "            nn.ReLU(),\n",
    "            nn.Dropout(),\n",
    "            nn.Linear(4096, 10)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.features(x)\n",
    "        x = self.classifier(x)\n",
    "        x = F.softmax(x, dim=1)\n",
    "        return x\n",
    "\n",
    "    def scan_structure(self):\n",
    "        x = torch.randn(size=(1,1,227,227)).to(device)\n",
    "        for layer in self.features:\n",
    "            x = layer(x)\n",
    "            print(f'{layer.__class__.__name__} output shape:\\t{x.shape}')\n",
    "        for layer in self.classifier:\n",
    "            x = layer(x)\n",
    "            print(f'{layer.__class__.__name__} output shape:\\t{x.shape}')\n",
    "\n",
    "net = AlexNet().to(device)\n",
    "# torchsummary.summary(net, input_size=(1,227,227))   # 查看网络结构\n",
    "\n",
    "epochs = 20\n",
    "lr = 0.1\n",
    "batch_size = 128\n",
    "\n",
    "transforms_ = transforms.Compose(\n",
    "    [\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Resize(size=(227,227))\n",
    "    ]\n",
    ")\n",
    "_, train_iter, test_iter = myutils.load_data_FMNIST('../data', batch_size, transforms_)\n",
    "\n",
    "optimizer = torch.optim.SGD(net.parameters(), lr)\n",
    "loss = nn.CrossEntropyLoss()\n",
    "\n",
    "myutils.train(net, train_iter, test_iter, epochs, optimizer, loss, True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "torch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
